Incremental Evolution of Complex General Behavior

Several researchers have demonstrated how complex action sequences can be learned through neuro-evolution (i.e. evolving neural networks with genetic algorithms). However, complex general behavior such as evading predators or avoiding obstacles, which is not tied to speci c environments, turns out to be very di cult to evolve. Often the system discovers mechanical strategies (such as moving back and forth) that help the agent cope, but are not very e ective, do not appear believable and would not generalize to new environments. The problem is that a general strategy is too di cult for the evolution system to discover directly. This paper proposes an approach where such complex general behavior is learned incrementally, by starting with simpler behavior and gradually making the task more challenging and general. The task transitions are implemented through successive stages of delta-coding (i.e. evolving modi cations), which allows even converged populations to adapt to the new task. The method is tested in the stochastic, dynamic task of prey capture, and compared with direct evolution. The incremental approach evolves more e ective and more general behavior, and should also scale up to harder tasks.